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tion  within  with  high  likelihood  (for  the  worst  case  behavior  of  the 
system) . 

We  have  very  few  assumptions.*  (1)  Tameness  is  required  of  a  process 
only  during  the  interval  it  is  wiping  to  communicate  (if  the  tameness  property 
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its  speed  arbitrarily  and  may  even  die.  (2)  The  processes  may  be  willing  to 
communicate  with  a  time  varying  set  of  processes  which  are  om> 
number.  There  are  no  probability  assumptions  about  system  behavior. 
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l. Our  communication  model  and  synchronization  algorithms  are  quite  robust. 
They  are  applied  to  solve  a  large  class  of  real  time  resource  synchronization 

problems ,  as  well  as  real  time  implementation  of  the  synchronization 
primitives  of* Hoars ’a  multiprocessing  language  CSP.<^ _ 


+  Note  that  we  do  not  us*  any  standard  high  level  synchronization  construct 
such  as  shared  variables  with  a  mutual  exclusion  mechanism.  If  we  did,  then  we 
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time  implementations  of  such  mechanisms  (in  fact,  there  is  no  bounded  time  imple¬ 
mentation  of  such  mechanisms  when  processes  run  on  different  processors).  We 
hope  in  the  future  that  our  techniques  rather  than  other  "standard”  but  inefficient 
synchronization  mechanisms  will  be  utilized  for  real  time  process  synchronization. 
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SUMMARY 


This  paper  considers  a  fixed  (possibly  infinite)  set  II  of  distributed 
asynchronous  processes  which  at  various  times  are  willing  to  communicate 
with  each  other. 

We  describe  probabilistic  algorithms  for  synchro. nixing  this  communica¬ 
tion  with  boolean  "flag”  variables,  each  of  which  can  be  written  by  only 
one  process  and  read  by  at  most  one  other  process.  The  use  of  flag  variables 
seems  as  to  require  the  fewest  assumptions  possible  without  considering 
specific  systems. f  A  process  is  considered  to  be  tame  over  a  time  interval 
A  if  its  speed  varies  within  certain  arbitrarily  fixed  nonzero  bounds. 

We  show  our  synchronization  algorithms  have  veal  time  response: 

If  a  pair  of  processes  are  mutually  willing  to  communicate  within  a 
time  interval  A  and  the  pair  are  tame  on  A,  then  they  establish  communica¬ 
tion  within  A  with  high  likelihood  (for  the  worst  case  behavior  of  the 
system) . 

We  have  very  few  assumptions:  (1)  Tameness  is  required  of  a  process 
only  during  the  interval  it  is  willing  to  communicate  (if  the  tameness  property 
is  violated  during  that  interval,  then  there  may  be  lower  probability  of 
successful  communication) ;  at  other  times  any  process  may  dynamically  vary 
its  speed  arbitrarily  and  may  even  die.  (2)  The  processes  may  be  willing  to 
communicate  with  a  time  varying  set  of  processes  which  are  only  bounded  in 
number.  There  are  no  probability  assumptions  about  system  behavior. 

Our  communication  model  and  synchronization  algorithms  are  quite  robust. 

They  are  applied  to  solve  a  large  class  of  real  time  resource  synchronization 

+  Note  that  we  do  not  use  any  standard  high  level  synchronization  construct 
such  as  shared  variables  with  a  mutual  exclusion  mechanism.  If  we  did,  then  we 
would  have  to  assume  an  implementation  of  such  a  mechanism  and  there  are  no  real 
time  implementations  of  such  mechanisms  (in  fact,  there  is  no  bounded  time  imple¬ 
mentation  of  such  mechanisms  when  processes  run  on  different  processors).  We 
hope  in  the  future  that  our  techniques  rather  than  other  "standard"  but  ineff icier., 
synchronization  mechanisms  will  be  utilized  for  real  time  process  synchronization. 


problems,  as  well  as  real  time  implementation  of  the  synchronisation 
primitives  of  Hoare's  multiprocessing  language  CSP. 

1  INTRODUCTION 

Recently,  [Rabin,  80],  [Lehman  and  Rabin,  81],  and  [Frances  and  Rodeh, 

80]  have  proposed  probabilistic  algorithms  for  a  number  of  synchronisation 
problems.  This  probabilistic  approach  (where  we  make  no  probabilistic 
assumptions  about  the  system  behavior,  but  allow  our  algorithms  to  make 
probabilistic  choices)  leads  to  considerably  sinpler  algorithms  (perhaps 
because  of  the  locality  of  their  decisions)  and  shorter  proofs  (perhaps 
because  the  proofs  cf  the  corresponding  deterministic  algorithms  had  to  con¬ 
sider  complex  situations  which  would  have  very  low  probability,  if  probabilis¬ 
tic  choices  were  taken) .  The  probabilistic  approach  may  also  lead  to  improve¬ 
ment  in  the  efficiency  of  synchronization  algorithms.  An  improvement  in 
space  efficiency  is  seen  in  [Rabin,  80].  We  demonstrate  here  that  a  consider¬ 
able  improvement  in  time  efficiency  can  be  made  by  probabilistic  synchronization. 

This  paper  takes  the  probabilistic  approach  to  synchronisation  of 
cormunication  in  a  network  of  distributed,  asynchronous  processes.  We  are 
interested  in  direct  interprocess  communication,  rather  than  packet  switching 
as  considered  in  [Tonag,  80]  and  [Valiant,  80].  Furthermore,  we  consider 
handshake  communication  (as  in  Hoare's  CSP),  rather  them  buffered  communica¬ 
tion  (which  is  very  easy  to  implement  by  message  queues) . 

Previously  [Schwartz,  80]  proposed  a  deterministic  synchronization 
algorithm  for  implementing  CSP  [Hoare,  78]  on  a  fixed  acyclic  distributed 
network.  Also  [Lynch,  80]  gave  a  related  algorithm  for  resource  synchroniza¬ 
tion  problems.  Both  algorithms  are  considerably  less  time  efficient  than  our 
proposed  algorithm  (for  a  specific  comparison  of  time  performance,  see 


Section  2,e~),  [Frances  and  Rodeh,  80]  also  propoca  a  probabilistic  solution 
to  synchronisation  of  conmuni cation,  but  make  no  consideration  of  the  time 
efficiency  of  their  solution. 

Our  paper  is  organised  as  follows:  As  is  usual  in  the  study  of  combina¬ 
torial  problems-,  we  state  the  problem  before  giving  our  proposed  solution.  We 
present  in  Section  2  a  model  for  distributed  communication  systems:  the 
mo'iel  ignores  the  details  of  message  transmission  but  gives  a  precise 
combinatorial  specification  (by  time  varying  graphs)  of  the  synchronization 
problem  of  interest.  This  model  also  allows  a  precise  definition  of  the 
relevant  complexity  measures  of  synchronization  algorithms,  such  as  response 
time.  Section  3  presents  our  synchronization  *  gorithms,  and  in  Section  4 
we  prove  various  properties  of  the  synchronization  algorithms  which  must  hold 
with  certainty,  regardless  of  probabilistic  choice.  Sections  5  and  6  give  a 
probabilistic  analysis  of  the  performance  of  our  algorithms.  We  have  taken 
considerable  effort  in  the  design  of  our  synchronization  algorithms  to  improve 
their  expected  time  performance.  Nevertheless,  our  algorithms  are  very  simple 
in  conception  and  practice.  The  Appendix  provides  a  real  time  implementation 
of  CSP. 

[Reif  and  Spirakis,  81C]  presents  a  further  application;  a  real  time 
resource  granting  system.  We  feel  these  applications  demonstrate  the  broad 
applicability  of  our  synchronization  algorithms. 

2.0  OUR  MODEL  FOR  A  DISTRIBUTED  COMMUNICATION  SYSTEM  (DCS)  AND  ITS  COMPLEXITY 

MEASURES 

Let  II«{1,2,...}  be  a  fixed,  (possibly  infinite)  collection  of  processes. 
We  assume  a  (global)  time  t,  on  the  nonegative  real  line  [0,°°] ,  whereby 
events  of  the  system  are  totally  ordered.  The  processes  of  II  are  aeyn- 
ohvonoUBi  their  speeds  may  dynamically  vary  arbitrarily  over  time  and  may  even 


be  0.  (Thus,  we  allow  processes  to  die.)  The  processes  have  no  access  to 
any  global  clock  giving  the  time.  (In  contrast,  [Arjomandi,  Fischer,  Lynch, 

81]  consider  synchronisation  problems  with  access  to  a  global  clock.) 

We  also  assume  a  global  oracle  <V  which  directs  the  willingness  of 
processes  to  communicate  with  each  other.  (Note  that,  in  applications  of 
DCS  occuring  in  practice,  no  such  oracle  sji  exists,  but  instead  each  process 
is  running  some  program  which  requires  from  time  to  time  communication  with 
other  processes.  An  implementation  of  the  DCS  synchronises  this  communica- 
tion.  The  oracle  <V  is  utilised  as  an  artificial  device  for  specifying 
worst  case  situations  of  our  system  where  communications  are  required  by  u/ 
to  be  made  at  times  most  difficult  for  our  implementation.) 

Intuitively,  each  process  i  wishes  at  various  times  to  communicate 
with  processes  in  IT— ii } .  All  communication  required  by  the  oracle  is 
implemented  by  i  rather  than  a  global  centralised  synchronisation  mechanism. 
Thus  system-wide  communication  is  implemented  by  a  distributed  scheduler,  the 
processes  of  II. 

The  formal  model  DCS  (for  Distributed  Communication  System)  described 
below,  has  been  designed  with  as  few  assumptions  as  possible  and  as  general 
as  possible.  We  are  not  concerned  with  the  values  of  the  messages  communicated 
between  the  processes,  but  instead,  with  simply  the  establishment  of  aorrmmi- 
aation.  This  allows  us  to  avoid  any  message  system  dependent  assumptions 
which  may  vary  for  any  given  application.  Also,  we  are  concerned  only  with 
direct  (two  way)  oormunioation  between  processes;  we  are  not  concerned  with 
packet  switching,  as  in  [Valiant,  80]  and  [Tonag,  SO], 

We  now  introduce  some  graphs  to  precisely  describe  the  DCS  model.  These 
graphs  allow  us  to  state  the  synchronisation  problems  precisely  as  combina¬ 
torial  problems  on  time  varying  graphs.  We  give  an  intuitive  description  of 
the  importance  of  these  graphs  as  they  are  defined. 
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N«  assume  a  possibly  infinite  undirected  graph  H,  the  connections 
graph  with  vertices  II,  and  undirected  edges  given  by  syimetric  antireflexive 
relation  **  e(HxII)-{(i ,i)  t  iell).  Then  i«^j  denotes  that  ieU  it  physic- 
oally  obit  to  oomrunioate  with  jell-UK  H  is  fixed  for  all  time  and  can  be 
considered  to  be  essentially  the  hardware  connections  between  processes  of  II. 
We  assume  H  has  finite  valence  (i.e.,  only  a  finite  number  of  processes  are 
connected  to  any  given  process  icTI) . 

Fo>  each  time  t>0,  we  assume  a  possibly  infinite  directed  graph  G^. 
the  willingness  digraph  with  vertices  II  and  directed  edges  given  by  relation 
•+  C  11*11.  Then  i  -£-*  j  denotes  that  iell  is  willing  to  oormunicate  with 
jdl-{i)  at  time  t.  (In  that  sense  we  say  i  is  the  source  and  j  is  the 
target).  We  require  that  i«^j  is  i  —£+  j  so  i  is  willing  to  communicate 
only  with  processes  which  i  is  able  to  communicate  with.)  Also,  let  i  «  -  ■»  j 
if  both  i  -£-*  j  and  j  i  .  Thus,  i«  t  ►  j  denotes  that  i,j  are  both 
willing  to  communicate  at  time  t.  For  each  time  interval  A  on  (0,“>) ,  let 
i  -j-*  j  if  i  -£-*  j  for  all  t€A,  and  let  i*-^~*  j  if  both  i  -j-*  j  and 

j  — i.  Thus  — r-*  and  *  denote  that  the  willingness  to  comnunicate  holds 

A  A  A 

over  time  intervals.  The  edges  of  departing  from  itlT  are  assumed  to  be 

stored  locally  at  i,  in  the  form  of  a  set  E^  whose  elements  are  the  names 
of  the  targets  of  i  at  time  t.  The  set  E^  is  specified  by  the  oracle 
and  read  only  by  i.  0 

Assumption  Al  We  assume  that  two-way  communication  between  any  two 
processes  i , jell  requires  only  one  step  of  i  and  j.  (Thus,  i,j  are 
assumed  to  communicate  in  short  "bursts.”) 


2. A.  Implementation  of  a  DCS 

An  t implementation  of  a  DCS  assign*  a  fixed  program  to  each  of  the 
processes  of  II.  Die  implementation  is  eyrmetric  if  the  programs  are  independent 
of  the  position  of  i  in  the  connections  graph  H. 

For  each  t>  0,  we  assume  a  (possibly  infinite)  directed  graph 
with  vertices  IT  and  directed  edges  given  by  relation  ell  *  IT.  Then 

i  j  denotes  i  opens  conrmnioation  with  j£H-ii}  at  time  t.  Let 

i  j  if  both  i  j  an<j  j  i.  Then,  i  4^^-*  j  denotes  i,j 

achieve  mutual  communication  at  time  t.  (Also,  we  extend  the  notation  to 

intervals  A  on  (0,«)  as  for  Gfc) . 

Assumption  A2  We  also  assume  that  if  i  j  and  not  i  — j, 

AAA  .  1  2 

for  some  t,  t^tctj*  i.e.,  the  oracle  *)4  can  withdraw 

willingness  to  coimunicate  only  after  communication  has  been  established. 

For  each  i.jCIT  such  that  i**j  we  assune  a  communication  port  flag 
P0RTi,j  <cont«>llea  by  process  i)  which  is  1  at  time  t  >0  if  i  j 

(i.e.,  process  i  has  opened  its  port  for  communication  with  j)  and  0  other¬ 
wise  (indicating  the  communication  port  from  i  to  j  is  closed) .  Thus, 

2  4^^-*  j  if  and  only  if  PORT^j  and  PORT^  ^  ere  simultaneously  1  at 
time  t.  We  assume  2-way  communication  between  i.j  is  possible  at  any  time 
both  PORT^j  and  PORT ^  ^ ^  are  simultaneously  1,  but  we  make  no  particular 
assumptions  (beyond  assumptions  Al,  A2)  about  this  communication. 

An  implementation  is  proper  if  it  satisfies  the  following  restrictions: 

Rl  i  j  only  if  i  j 

R2  i8  a  (partial)  matching:  if  i  j  then  not 

i  j'  for  any  j 

Note  that  Ri  implies  that  the  poller  of  i  opens  communication  with 
j  only  if  i.j  are  simultaneously  willing  to  communicate.  R2  inplies 
that  i  does  not  communicate  with  more  than  one  process  at  a  time. 

It  is  standard  in  the  study  of  combinatorial  algorithms  to  specify 
the  combinatorial  problem  before  giving  algorithms  for  the  solution.  We  have 


precisely  described  the  problem  of  determining  a  DCS  incrementation  as  a 
combinatorial  problem  on  dynamic  graphs.  Later  we  shall  propose  two  imple¬ 
mentations  satisfying  both  these  restrictions.  Still  another  implementation 
is  described  in  [Reif,  Spirakis,  813}. 


2.B.  Global  State  of  the  DCS 

For  each  t  >  0,  let  R^  be  a  mapping  from  II  to  the  reals  giving  the 
speed  of  each  process  of  II-  at  time  t.  We  assume  the  speed  schedule 
R  »  iRt | t  _>  0}  is  chosen  by  an  oracle  u/  (our  scheduler's  worst  "enemy") 
at  time  t  ■  0.  Also,  we  assume  for  each  t  ^  0,  jf  chooses  (for  the 
processes  of  II)  the  willingness  digraph  G^  at  time  t.  (Thus#  G^  may 
vary  dynamically  in  time,  depending  on  the  choices  of  the  oracle  %rf) .  How¬ 
ever,  for  each  t  0,  the  digraph  Mt  is  defined  by  the  processes  of  II, 
(which  attempt  a  distributed  synchronization  of  the  DCS,  depending  on  our 
given  implementation) .  In  addition,  we  allow  the  processes  of  II  to  make 
independent  probabilistic  choices. 

Let  Lfc,  the  luck  up  to  time  t,  be  the  probabilistic  choices  made  by 
the  processes  of  H»  up  to  time  t.  Then,  the  global  system  state  at  time 
t  is  given  by 


«  <Rt,Gt;Mt,Lt,t> 

and  the  global  history  up  to  time  t  is 

I\  -  {E*.|°  <  t*  <  t} 

w  - 


Thus,  we  have  a  probabilistic  multiplayer  game  of  incomplete  information, 
where  the  omnipotent  oracle  plays  against  the  team  of  processes  of  II 


e-  ' 


{which  have  only  incomplete  information  on  the  current  state  of  the  system). 

We  wish  measures  of  the  success  of  the  processes  of  IT. 

2.C.  Complexity  Measures  on  DCS  Implementations 

In  the  following  we  assume  that  there  exists  a  given  ^ixed  integer 

constant  v  >  0  such  that  Vi€ll,  Vt  0,  the  outdegree  of  i  in  (i.e, 

the  cardinality  of  { j  J  i  “£“*  j})  is  bounded  above  by  v. 

Let  process  i  be  tare  on  the  interval  A,  if  its  speed  (steps  per 

real  time  unit)  of  i  at  times  t  on  A  is  on  the  interval  —  ,  —  — ] 

max  min 

where  r  .  ,  r  are  fixed  real  constants  and  0  <  r  .  <  r.  „  (Without 

mm  max  min  —  max. 

loss  of  generality  we  assume  that  r  /r  .  is  an  integer) .  A  step  consists 
of  either  an  assignment  of  a  variable,  a  test,  a  logical  or  arithmetic 
operator,  or  a  no-op. 

We  shall  not  assume  that  processes  are  tame  at  all  times.  Our  DCS 
implementation  will  be  proper  irregardless  of  whether  processes  are  tame  as 
long  as  their  speeds  are  nonzero. 

Let  processes  i,j  have  successful  communication  at  interval  A  if 
i-T*  j  and  A  contains  at  least  one  step  of  both  i  and  j.  We  say  A 
is  a  response  interval  for  processes  i,j  if  A  is  a  maximal  time  interval 
such  that 

(1)  i^j, 

(2)  i,j  are  both  tame  on  A,  and 

(3)  i,j  have  successful  communication  at  most  just  at  the  end  of  A, 
if  at  all. 

(Note  that  since  A  is  maximal,  either  i,j  were  not  mutually  willing  to 
communicate  immediately  before  A,  or  A  begins  at  time  0.  Also  note  that 
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an  oracle  can  make  a  response  interval  infinite  if  i,j  do  not  ever  have 
successful  communication.) 

Let  the  response  ti>ne  of  a  DCS  implementation,  for  any  oracle  a/, 
be  the  random  variable  giving  the  length  of  a  response  interval. 

Let  t  »  maximeanfi^  )/all  oracles  ,V}.  For  each  e,  0  £  e  £  1,  let  the 
t-error  response  t(e)  (note:  this  is  a  function,  not  a  random  variable)  be 
the  least  upper  bound  on  the  inverse  of  the  cumulative  distribution  function 
of  Thus,  if  we  have  a  finite  interval  A»  |A|  >  t(e)  and  any  two 

processes  i,j  which  are  tame  on  A,  for  all  oracles  «V,  i  *~£*  j  implies 

1, j  have  successful  conmunications  sometime  within  A,  with  probability 

>  l-£.  Note  that  time  response  as  defined  above  for  pairs  of  processes  also 
holds  for  communication  between  sets  of  processes.  Suppose  we  have  finite 
sets  of  processes  JI^»  H2  c  II  such  that  for  all  i  in  ni  and  all  j  in 
n2,  i *  j  for  the  same  finite  interval  A  of  length  £  x'c).  Then,  for 
all  oracles  u/  and  all  i€  and  j€  ^  »  i  and  j  have  successful  communicatio 
sometime  within  A  with  probability  >l-e  for  each  pair  of  processes. 

This  implies  a  very  robust  type  of  fairness.  Each  pair  of  JI^  IIj 
are  guaranteed  _>l-e  probability  of  successful  communication  within  A» 
independent  of  the  success  of  communication  by  other  processes. 

The  DCS  implementation  is  real  time  if  for  all  e,  0  <  e  <  1,  x(e) 
is  a  constant  dependent  only  cm  v,  assumed  to  be  a  constant  upper  bound  on 
the  outdegree  of#  vertices  of  Gt)  and  t  is  bounded  above  by  a  fixed 
constant.  (Note  that  x  <  T(e)  •  e  (Je) . 

2. D.  Insisting  DCS  Implemei  Nations 

We  also  consider  the  cases  w.iore  any  given  process  i&I  may  assign 
a  priority  to  the  processes  j€Jl"{i}  which  i  wishes  to  communicate  with. 
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in  the  simplest  cape,  i  distinguishes  the  first  target  of  communication, 
say  E^(l),  which  i  insists  on  communicating  with  (process  i  may  eventually 
communicate  with  the  other  processes  of  E^ ,  but  i  insists  on  communicating 
with  E^(l)  with  highest  priority).  For  each  t  >_  0,  is  the 

relation  on  II  *  II  such  that  Vi,j£TI  i  — £-»*  j  iff  e^(1)  *  j  at  time  t 
(also  let  i  ‘■J**  j  if  i.  -£•'  j  Vt€A)  . 

We, say  A  is  an  insisting  response  interval  for  i,j  if  A  is  a  maximal 
interval  such  that 

(1)  i  -J*'  j  and  j  -J*  i 

(2)  i,j  are  both  tame  on  A,  and 

(3)  i,j  have  successful  communication  at  most  just  at  the  end  of  A, 
if  at  all. 

(Note  that  only  the  first  process  has  to  distinguish  the  ether  as  the  first 
target.) 

Let  the  insisting  response  time  of  a  DCS  implementation  be  the  random 
variable  T«V'  for  each  oracle  giving  the  length  of  an  insisting  response 
interval.  Let  T*  =  max{mean(T'  )/all  oracles  o</}.  For  each  e,  0  <  z  <  1 
let  the  c~ error  insisting  response  t' (c)  be  the  least  upper  bound  on  the 
inverse  of  the  cumulative  distribution  function  of  T  .. 

Thus,  if  we  have  a  finite  interval  A,  |A|  >  t* (£)  and  any  two  processes 
i,j  which  are  tame  on  A,  for  every  oracle  (i  j  and  j  i) 

implies  i,j  have  successful  communication  sometime  within  A,  with  probability 
>  1-e. 

The  DCS  implementation  has  real  time  insisting  response  if  for  all  e, 
o  <  e  <  3,  T ' (t)  is  a  constant,  independent  of  any  parameter  of  H  (exeept  v) 
and  t*  is  bounded  above  by  a  fixed  constant. 

It  is  useful  to  observe  that,  given  T  *  (C ) ,  any  given  process  i£JI  may 
determine  (with  any  given  probability)  whether  any  other  process  j€JI-{i}  is 
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willing  to  communicate  with  i  over  a  given  time  interval  in  which  both 

1, j  are  tame. 

PROPOSITION  2.1.  Let  a/  be  any  oracle  and  A  be  any  time  interval 
of  finite  length  >  T'(e).  Suppose  i,j  are  tame  on  A.  If  i  j 

but  there  is  no  t€A  such  .that  i  ***  j  ,  then  j  is  not  willing  to 
communicate  with  i  sometime  during  A,  with  probability  >l-e.  This  propo¬ 
sition  may  be  used  for  timing  out  insisting  requests  to  communicate  with  a 
Specific  process. 

(Note:  Suppose  we  are  given  a  process  j,  a  set  of  processes 
c  H  and  an  interval  A  T'(e)  such  that  for  all  iCTI^  , 

i  j  ar»d  j  i.  Assume  also,  all  processes  in  U  {j}  are  tamo 

on  A.  Then,  for  each  iCTI^  and  for  all  oracles  a/,  i,j  have  successful 
communication  sometime  with  A,  with  probability  >l-e.) 

2. E.  Results  and  Previous  Work 

The  primary  results  for  this  paper  are: 

There  is  a  veal  time  implementation  of  DCS  such  that 

_  2 

(1)  the  worst  case  mean  response  T  is  0{v  ); 

2 

(2)  the  e-error  response  T(e)  is  0(v  log(l/e)). 

Also,  there  is  a  veal  time  implementation  of  DCS  such  that 

(1)  worst  case  mean  insisting  response  T*  is  0(v); 

(2)  the  e-error  insisting  response  T'(e)  is  0(v  log(l/e)). 

These  results  follow  from  a  single  general  theorem  of  Section  4. 

Our  implementations  are  proper,  symmetric,  and  are  completely  independent 
of  the  connection  graph  H  (H  may  be  any  finite  or  infinite  graph  with 
finite  valence) .  Our  innovation,  which  results  in  real  time  response,  is  to 
allow  processes  to  make  probabilistic  choices. 
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The  best  previous  result  is  due  to  [Schwartz,  80]  and  is  restricted  to 
the  case  il  is  finite  and  its  edges  can  be  directed  to  form  a  digraph  H* 
which  is  acyclic.  Let  XM  be  the  minimum  vertex  coloring  of  any  such  P'. 
Essentially,  the  technique  of  [Schwartz,  80]  is  to  color  H*  and  order  the 
precedence  of  message  transmissions  by  the  coloring.  Delays  in  message 
transmissions  can  be  as  long  as  '/(H)  since  chains  ofprocesses  (of  length 
X(H)  can  be  formed  in  which  each  process  waits  for  the  next  to  reply.  So 
the  deterministic  DCS  implementation  of  [Schwartz,  80]  has  insisting 
response  time  T*  lower  bounded  by  v»x(H).  Note  that  his  implementation  is 
not  real  time,  since  in  general  xW  is  of  order  | T!  |  .  Also,  his  DCS 
implementation  is  not  ay  metric,  since  processes  are  required  to  know  their 
color  in  H’ . 

Also  [Lynch,  80]  gives  a  solution  to  a  distributed  resource  allocation 
problem,  which  in  [Reif,  Spirakis,  81C]  is  adopted  to  yield  a  DCS  implementation 
with  response  time  .  (In  [Reif,  Spirakis,  8lc]  we  show  that  a  class 

of  generalized  resource  allocation  problems  related  to  those  of  [Lynch,  80] 
may  be  efficiently  solved  by  our  DCS  implementation.) 

[Francez,  Rodeh,  80]  proposed  a  probabilistic  synchronization  algorithm 
which  can  be  considered  to  be  a  DCS  implementation.  An  important  difference 
between  our  implementation  and  theirs  is  that  in  the  responding  phase,  in  our 
algorithms,  each  process  responds  to  all  processes  to  which  it  is  willing  to 
communicate,  while  in  [Frances,  Rodeh,  80]  only  one  process  is  considered  at 
a  time.  Although  [Frances,  Rodeh,  80]  make  no  explicit  timing  assumptions, 
they  do  assume  that  setting  and  resetting  of  shared  variables  takes  only  a 
negligible  time  compared  to  the  waiting  time  of  processes,  which  is  a  much 
stronger  assumption  than  ours.  The  careful  consideration  of  timing  in  our 


paper  is  crucial  to  our  achievement  of  real  time  response  (see  also  the 
analysis)  and  such  timing  considerations  were  essentially  not  considered  in 
any  previous  papers  on  synchronization.  Also,  we  utilize  probabilistic 
choice  in  new  ways  than  those  utilized  in  [Francez,  Rodeh,  80].  In  particular 
we  utilize  random  waits  to  ensure  that  the  oracle  cannot  make  the  behavior 
of  waiting  processes  depend  on  choices  of  partners  made  by  other  processes 
(see  the  analysis  of  the  insisting  algorithm). 

3  OUR  IMPLEMENTATION  OF  A  DCS 

To  implement  a  DCS,  we  must  give  an  algorithm  for  each  process  in  H. 

We  present  here  two  such  implementations.  Both  satisfy  restrictions  R1,R2 
required  by  proper  implementations,  and  both  are  symmetric:  Each  process  has 
the  same  algorithm  regardless  of  its  position  in  the  graph  H.  Processes 
have  Algorithm  1  in  our  "noninsisting"  implementation,  and  Algorithm  2  in  our 
"insisting"  implementation.  We  show  in  Section  4  that  both  implementations 
have  real  time  response. 

Each  program  variable  X  of  the  system  may  be  written  by  exactly  one 
process  ie  II  and  either  X  is  read  by  only  one  other  process  j€JI  —  {i } 

(in  this  case  x  is  a  flag  from  i  to  j)  or  X  is  looal  to  i  (x  is 
read  only  by  i) . 

Our  following  description  of  the  DCS  implementations  will  be  given  top- 
down  with  a  high  level  specification  of  the  algorithms  given  first  and  then  a 
specification  of  the  procedures  ASK, RESPOND  which  they  call.  (The  procedures 
ASK, RESPOND  utilize  numerous  flag  variables  which  are  irrelevant  to  the  super¬ 
fluous  understanding  of  our  algorithms.)  Also,  before  giving  the  formal  speci 
fications  of  any  algorithm  or  procedure,  we  give  in  English  an  informal 


description  of  its  sc'. ‘one.  The  actual  formal  algorithms  have  been  written 
carefully  to  satisfy  certain  timing  restrictions  required  by  our  analysis  to 
achieve  real  time  response. 

In  both  algorithms,  each  process  repeatedly  throws  a  fair  coir  and  then 
executes  a  phase.  Each  phase  is  either  asking  or  responding  and  is  chosen  by 
the  coin  throw  with  a  probability  1/2. 

Algorithm  1  (Non-insisting) 

In  the  responding  phase,  process  i  repeats  a  loop  v  times.  On  each 
iteration  of  the  loop,  process  i  chooses  at  random  a  process  j  from  the 
processes  i  is  willing  to  communicate  with,  and  executes  procedure  RESPOND^ (j). 
This  procedure  takes  constant  number  of  steps  (namely  cR)  and  during  it 
process  i  tests  a  flag  to  determine  if  j  is  currently  willing  to  talk  to 
i  and  then  sets  a  flag  to  determine  if  j  pays  attention  to  i  (these  tests 
have  the  form  of  a  handshake) .  If  so,  processes  i  and  j  synchronize  their 
steps  and  then  both  open  communication  to  each  other.  In  either  case,  i 
repeats  the  loop  until  the  responding  phase  finishes. 

In  the  asking  phase,  process  i  chooses  only  once  at  random  a  process  j 
to  which  i  is  willing  to  communicate  with,  and  the  i  executes  procedure 
ASK^(j).  This  procedure  takes  cft  -  cR»v  steps  (so  that  both  phases  take 
exactly  the  same  number  of  steps.  As  a  consequence,  process  i  is  in  each 
phase  half  of  the  time  on  the  average.  This  is  important  to  the  analysis). 
During  procedure  ASK^(j),  process  i  raises  a  flag  to  show  to  j  that  it  is 
willing  to  communicate  currently  with  j,  and  then  pays  attention  to  j  for 
a  limited  number  of  steps  to  test  if  j  responds  to  the  atten\pt  and  wants  to 
proceed  in  communication.  If  so,  then  processes  i  and  j  synchronize 
their  steps  and  then  both  open  communication  to  each  other.  If  not,  then  i 
finishes  its  current  phase  by  clearing  its  flags. 


Algorithm  2  (Insisting) 


Each  process  executes  forever  the  following  loop:  It  begins  by 

choc3ing  a  random  integer  w  from  {0, 1, 2, . . . , c. } .  It  then  waits  for  w 

A 

steps  and  then  it  chooses  with  probability  1/2  to  execute  a  respond  phase 
or  a  (modified)  ask  phase.  The  respond  phase  is  identical  to  that  of  Algo¬ 
rithm  1.  However,  in  the  modified  ask  phase,  process  i  chooses  the  dis¬ 
tinguished  first  process  E,, (1)  as  the  process  to  which  it  will  apply  the 
procedure  ASK^.  After  executing  one  or  the  other  of  the  phases,  process  i 
then  waits  for  CA“W  steps.  (This  guarantees  that,  at  c.ny  time  t,  a  process 
is  not  waiting  with  probability  1/2,  and  that  a  process  is  asking  (given  it 
is  not  waiting)  with  probability  1/2) . 


We  now  give  Algorithms  1  and  2  in  full  detail. 

Algorithm  1  (noninsisting  implementation) 

Program  for  process  i€II 

INITIALIZE^ (  ) ; 

WHILE  TRUE  DO 
BEGIN 

L2:  CHOOSE  a  random  be{0,l} 

IF  b  -  0  THEN 
BEGIN 

COMMENT:  respond  phase 
L3:  FOR  x«l  to  v  DO 
BEGIN 

CHOOSE  at  random  j  €  E^ 

RESPONDi  ( j ) * 

END 

END 

ELSE 

BEGIN 

COMMENT:  ask  phase 

L4:  CHOOSE  at  random  j  €  E^ 

AS^  (j ) 

END 

END 

OD 
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Algorithm  2  (the  insisting  implementation) 

Program  for  process  i  €  J! 

INITIALIZE^ (  ) 

WHILE  TRUE  DO 
BEGIN 


Lis  CHOOSE  w  at  random  from  {0,l,...,cA) 
DO  w  no-ops 

L2:  CHOOSE  a  random  be{0,l} 

IF  b  «  0  THEN 
BEGIN 

COMMENT;  respond  phase 
L3:  FOR  X«1  to  V  DO 
BEGIN 

CHOOSE  a  random  j  €  E^ 
RESPOND^;}) 

END 

END 

ELSE 

BEGIN 

COMMENT ;  ask  phase 
L4:  ASKi(Ei(l)) 

END 

DO  cA  -  w  no-ops 

END 

OD 


3. A.  Intuitive  Description  of  the  Procedures  ASK, RESPOND 

The  procedures  ASK^,  RESPONDi  are  utilized  by  both  algorithms. 

For  each  i,j€II  such  that  i«*j,  there  are  three  flags  (boolean 
variables)  ,  A^»  which  are  written  only  by  i  and  read  only  by  j. 

(1)  Flag  Q^j!  Just  before  each  phase,  *  0.  Then  i  asks  j  by 

setting  to  1  in  the  ask  phase.  is  reset  to  0  before 

the  end  of  the  ask  phase. 

(2)  Flag  A^ :  Just  before  each  phase,  A^j  “0.  If  i  is  in  the 

answer  phase  and  detects  *  1  (indicating  j  "asks"  i)  then 

i  answers  j  by  setting  A^  «■  1.  Before  the  end  of  the  answer 
phase,  i  resets  A^ 


to 


0 


. . .  ..  .  - - 


-17- 


(3)  Flag  B  ji  This  variable  is  set  to  0  by  i  only  during  the 

j 

"Matching  windoo"  which  is  the  interval  when  i  is  in  the  asking 
phase  anc  is  watching  for  an  answer  (A^  *  1)  from  j.  At  all 

other  times,  is  set  to  1  to  indicate  i  is  blind  to 

answers  by  j . 

Another  flag  PORT^  is  utilised  by  the  low  level  procedure  OPEN-COM 
to  specify  the  state  of  the  communication  port  from  i  to  j.  As  defined  in 
Section  2,  i  j  iff  PORT^  -  1  at  time  t.  (OPEN-COM  is  called  by 

ASK^  and  RESPOND^  as  the  final  act  in  a  successful  communication  attempt.) 

If  process  i  executes  ASK^ (target)  then  it  first  sets  a  flag  variable 
Q  ^ , target  to  *  ^to  indicate  to  j  that  it  asks)  and  sets  another  flag 

B.  .  .  to  0  <to  indicate  to  j  that  it  pays  attention  to  it,  i.e.,  i  is 

not  blind  to  answers  by  j).  It  keeps  these  flags  raised  for  at  most  a  constant 
number  c_  steps  and  during  these  steps  it  continuously  examines  the  flag 
A.  .  .  (the  ans  /er  flag  of  target) .  If  the  interval  finishes  with  no  answer 
from  target,  then  i  first  sets  target  to  *  (to  *how  that  it  stops 

paying  attention  to  target)  and  then  it  sets  target  to  ®  to  ®fcp  t^ie 
question.  This  order  of  actions  guarantees  that  process  target  will  interpret 
correctly  what  it  sees  fro..i  the  flags  of  i. 

If  i  gets  an  a.:s>./er  from  j  (that  is,  if  target  i  *s  set  to  ** 
during  the  (previously  discussed)  cB  steps,  then  i  firei-  sets  target 
to  0  (but  keeps  B,  .  to  its  current  value  to  ii  Jicate  that  it  con- 

x f 

tlrues  to  pay  attention  to  j).  Process  i  waits  until  target  also  zeros 
its  flag  *target  ^  and  then  Process  i  calls  OPEN-COM^  (target)  immediately. 
As  the  analysis  shows,  the  events  leading  to  this  call  guarantee  that  communica¬ 
tion  is  achieved  between  i  and  target  during  the  execution  of  OPEN-COM, 
assuming  i  and  target  are  tame  (we  do  not  use  a  handshake  protocol  within 
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open-COM  since  unfortunately  our  timing  constraints  would  be  violated  if  we 
were  to  require  that  communication  between  i  and  target  be  successful 
irregardless  of  whether  they  are  tame) .  At  the  end  of  OPEN-COM,  i  sets 
Bi  target  to  *  that  it  stops  paying  attention  to  target)  and 

exits  procedure  ASK^. 

If  process  i  executes  procedure  RESPOND^  (as leer) ,  then  it  first 
examines  if  QagJter  i  ii  1  (i.e.,  if  asker  is  interested  in  communicating 
with  i) .  If  so,  then  i  sets  A^  M)ter  to  1  and  waits  until  process 
asker  seros  its  question  flag  (this  is  the  "handsl  ake"  technique).  When  this 
happens,  then  i  tests  Bas)ter  ^  to  8ee  process  asker  still  pays  atten¬ 
tion  to  i.  If  not,  then  i  seros  its  answer  flag  A^  asJt#r  *nd  exits. 

Else,  i  knows  that  asker  waits  for  step  synchronisation  and  conmunication. 
So,  i  zeros  its  flag  Ai  as)ier  and  calls  OPEN-COf^  (asker).  The  analysis 
shows  that  the  events  leading  to  this  call  guarantee  that  communication  will 
be  achieved. 

We  now  introduce  some  terminology  and  then  develop  the  algorithms  in 
full  detail. 

A  process  i  is  in  the  asking  mode  when  it  executes  procedure  ASK  ( j ) , 
and  it  is  in  the  responding  mode  when  it  executes  the  procedure  RESPOND,  if 
1  is  executing  ASK(j)  and  B^*0  then  i  is  in  a  watching  window  for 
process  j  else  i  is  blind  with  respect  to  j.  we  say  i  is  answered  by 
j  if  i  is  in  its  watching  window  for  j  and  i  exits  locy  A3  with  a»l. 

A  phase  of  the  algorithms  consists  of  the  steps  between  random  choices  of  the 
variable  b  €(o,l}.  If  b»0  the  process  is  in  an  answering  phase  and  else 
it  is  in  an  asking  phase. 

We  have  not  elaborately  commented  on  our  procedures  because  of  the 
extensive  informal  description  preceding  them. 
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The  variables  of  process  i  are  initialised  as  follows « 

INITIALIZE. (  ); 

BEGIN  1 

for  all  j  €  H  such  that  i**j  dc 
BEGIN 
Qij-Nl 

Aij^ 

Bij'"1 

PORT^+O 

END 

END 

In  the  following  two  procedures,  we  assume  a  register  CURSTEP  which  gives 
the  current  number  of  the  steps  executed  by  process  i  since  it  was  last  zeroed. 
(CURSTEP  ia  assumed  here  only  as  a  convenience,  it  is  clear  that  we  could 
substitute  instead  a  new  variable  that  is  incremented  on  every  step  of  the 
original  Algorithm.) 

We  have  made  extensive  use  of  time  outs  to  guarantee  that  the  number  of 
steps  of  the  execution  of  procedures  RESPOND,  ASK  are  each  always  exactly 
the  same.  (This  is  crucial  to  our  proof  of  real  time  response.) 

We  define  the  constants  appearing  in  the  procedures! 


Let  c. 


max 


6  +  20  -7 — —  i  this  will  be  precisely  the  number  of  steps  always 
min 


required  by  procedure  RESPOND. 

Let  cj^  *  cp  v  f  this  will  be  the  number  of  steps  required  by  procedure 


ASK. 


Let  cB  »  c^  -  (6  +  20  — )  i  this  is  the  number  of  steps  required  for 

min 


a  watching  window. 


Let  c  -  2  +  3 

P  r 


max 


min 


l  this  is  the  number  of  steps  required  in  procedure 


0PEH-C0M. 


Let  c  *  c  -  c  -  2;  this  constant  is  used  in  our  algorithms. 
D  A  P 
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PROCEDURE  ASK. (target) 
local  a 


BEGIN 


Alt  CURSTEP**0 


h2t  fii, target^1 


D .  .  .^V 

i ,  target 

COMMENT t  Begin  watching  window  for  target 

A3t  WHILE  CURSTEP  <  cfi  AND  a-0  DO  *4‘Atarget>i 

IF  CURSTEP  >  C  AND  a«0  THEN  B.  .  .  «-l 

—  —  B  -  - —  target 

®i , target*-0 
IF  a-1  THEN 
BEGIN 


A4:  WHILE  (A„_  .  .-0  AND  CURSTEP  <  C 

-  target ,  i  -  D 


A5:  IF  a«0  AND  CURSTEP  <  CD 


AND  a-1)  DO  t-A,.,^^ 


THEN  OPEN-CON, (target) 
-  1 


COMMENT:  End  watching  window  for  target 

B 

i, target 

WHILE  CURSTEP  <  c  DO  no-op 


PROCEDURE  RESPOND^ (asker) 
local  q 

BEGIN  CURSTEP^O 
<r*^asker,i 


Bl:  IF  q»l  THEN 
BEGIN 

*i,  asker  *■  1 

B2:  WHILE  (CURSTEP  <  CD  AND  q*l)  DO  q*Cagker  ^ 

T*-  (q  OR  asker  .  1  OR  CURSTEP  >  C  ) 

—  D 

B3:  A.  .  *0 

i,  asker 


IF’iq  THEN  B4:  OPEN-COM^ (asker) 


B5:  WHILE  CURSTEP  <  CR  DO  no-op 


END 
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PROCEDURE  OPEN-COMi ( j ) 

BEGIN 

PORT^l 

DO  Cp-2  no  ops 

PORTj^+O 

END 


4. A.  Correctness  Properties  of  the  Algorithms  which  Hold  with  Certainty 


l 

i 

k 

h 


1 1 


i 


Our  algorithms  are  probabilistic  and  therefore  some  of  their  properties 
(such  as  response  time)  only  hold  with  a  certain  probability ,  and  not  with 
certainty.  A  probabilistic  analysis  of  these  properties  is  given  in  the  next 
sections.  However,  in  this  section  we  prove  properties  of  the  algorithms 
which  hold  with  certainty,  regardless  of  probabilistic  choice.  We  show 
restrictions  Rl  ,R2  are  satisfied  by  our  implementati  >ns,  and  thus  they  are 
proper.  (Of  course,  we  assume  either  all  the  processes  in  II  execute 
Algorithm  1,  or  they  all  execute  Algorithm  2.) 

LEMMA  4.1.  For  both  algorithms, 

i  j  only  if  i  j. 

Proof .  Process  i  calls  OPEN-OOM^(j)  and  opens  its  channel  to  j 
only  if  either  (a)i  was  executing  an  asking  phase  and  exited  the  loop  A3 
with  a**l  or  (b)i  was  executing  a  respond  phase  and  exited  the  busy  wait 
B2  with  B.  . *0.  in  both  cases,  i  was  willing  to  communicate  with  j  in 

j 

the  start  of  the  execution  of  its  phase,  since  i  asks  (or  responds)  only  to 
processes  it  is  willing  to  communicate  with.  So,  i  — j  where  t*  was 
the  time  of  start  of  i's  phase.  By  assumption  (A2)  then,  i  j. 

In  case  (a) ,  a*l  means  that  j  responded  by  setting  A.  .  to  1  to 
i's  question.  So,  j  1  for  some  t"  <  t  and  by  assumption  (A2), 


.  >V  ■*«£*.*  _■ 


•  . .  vv.^.-s 


*  1* >, V* f >1, {;*■* |*y*  V>  (  *;•;/■«  1 1;?*/*!  :’>•  "v:vi 
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In  case  (b) ,  j  was  the  process  setting  Q.  .  to  1  at  the  beginning 

J  t  * 

of  i’s  phase.  Hence  j  i  and,  by  (A2)  ,  j  ♦-£-»  i. 

In  both  cases,  i  j  *  i  j  o 


t 


LEMMA  4.2.  For  both  algorithms, 

.-is  a  partial  matching. 

Proof.  Since  each  process  opens  communication  to  at  most  one  process 
each  time,  (this  is  so  since  the  programs  in  both  algorithms  are  sequential 
and  each  neighbor  is  asked  or  responded  to  separately)  ,  the  relation  -* 

is  one  to  one.  Hence 


cannot  be  more  than  a  matching. 


COROLLARY  4.1.  Both  algorithms  give  a  proper  implementatir n  of  DCS. 
Proof.  By  the  Lemmas  4.1,  4.2  both  R1,R2  hold. 


4.B.  Timing  Lemmas  Which  Hold  With  Certainty 

LEMMA  4.3.  For  both  algorithms,  if  i  is  in  its  watching  window 

for  j  and  is  answered  by  j,  then  i,j  have  successful  communication, 

within  20  steps  of  the  slowest  of  i,j,  given  both  i,j  are  taiue. 

Proof.  If  i  exits  the  A3  loop,  then  (since  no  process  but  j  can 

assign  to  A.  .)  at  the  same  time  j  must  be  executing  RESPOND. (i)  at  the 
J  3 

B2  loop.  Process  i  will  arive  at  A4  within  7  of  its  steps  and  will 
have,  by  then,  set  to  0.  Within  7  rmax/rin£n  steps  of  j,  j  will 

have  exited  the  B2  loop.  Also  at  this  time,  the  assumption  that  i  exits 
the  loop  A3  with  a=l  implies  that  B^=0.  So,  j  will  arrive  at  B3 
and  set  A.  ,  to  0  in  at  most  7  of  its  steps  from  the  time  it  exited  the 

j  •  1 

B2  loop.  Within  rmax/rmin  steps  of  i,  process  i  exits  the  A4  loop. 
Then,  within  two  of  its  steps  i  will  call  OPEN-COM^(j)  and  within  one 


of  j's  steps  j  will  call  OPEN-COM^  (i) .  Note  that  both  i,j  will  set 

their  respective  £ort  flags  PORT^,  PORT^  to  1  within  one  step  of  the 

slowest  process  (or,  within  at  most  rIftax/rmin  *teP8  of  the  fastest).  They 

keep  their  ports  open  for  c_-2  *  3  r  /r  .  steps  each.  This  implies 

*  m&x  min 

that  both  processes  will  overlap  for  at  least  2  rmax/r]T1^rj  *  rmin  *  2  rmax 
time,  guaranteeing  at  least  1  step  overlap  of  both  processes.  Thus,  i,j 
have  successful  communication.  Note  that  OPEN-COM  takes  cp  steps.  The 
Lemma  follows  by  counting  steps.  o 


.  AAA  . 

LEMMA  4.4.  For  both  algorithms,  if  i,j  tame  on  A  and  Sr  ^  j 
for  a  maximal  interval  A,  then  A  contains  a  step  of  both  i  and  j,  but 

| A |  ■=  0(1).  (This  ensures  that  A  is  just  long  enough  for  i  and  j  to 

communicate.) 

Proof.  i*/^Vi  implies  i-^^j  •  The  only  sequence  of  events  leading 

to  that  is  the  sequence  in  which  one  of  i,j  is  in  its  watching  window  for 

the  other  and  is  answered  by  the  other.  By  Lemma  4.3  then,  A  contains  a 

step  of  both  i,j.  Also,  by  Lemma  4.3,  |A|  <  20  r  ^/r  .  o 

—  max  min 


LEMMA  4.5*.  For  both  algorithms,  if  i,  j  are  tame  on  A  and  i 
for  a  maximal  interval  A,  then  j  for  some  A'  c  A.  Furthermore 

i , j  have  successful  communication  during  A'. 

AAA 

Proof.  The  only  sequence  of  events  leading  to  i  ^  ^  j  is  the 
sequence  in  which  one  of  i,j  was  in  its  watching  window  for  the  other  and 
is  answered  by  the  other.  By  Lemma  4.3,  3  A'  c  A  such  that  i,j  have 

successful  communication  during  A'.  D 


Note:  Lemma  4.5  means  that  a  tame  process  never  opens  its  channel 
to  another  tame  process  without  communicating  with  it. 
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In  the  following  lemma,  we  do  not  necessarily  assume  i  is  tame. 

Lemma  4.6.  if  i  £  II  executes  procedure  ASK,  then  precisely  cR  steps  of 
i  are  required  for  the  execution  of  this  procedure.  Execution  of  RESPOND 
by  i  requires  precisely  cR  steps  of  i.  Also,  each  phase  of  Algorithm  1 
requires  exactly  cft  +  2  steps  and  each  phase  of  Algorithm  2  requires 
2c A+  2  steps. 

Proof.  By  observation  of  timeouts  within  the  procedures  ASK  and  RESPOND. 
Corollary  4.2.  Let  cv  be  the  number  of  steps  required  for  each  phase 
(c  **  cR  +  2  for  Algorithm  1  and  c  «  2cR  +  2  for  Algorithm  2,  as  implied 
by  Lemma  4.6).  Then  the  maximum  time  required  for  each  phase  is  <cvrm^y « 

5.  PROBABILISTIC  ANALYSIS  OF  THE  RESPONSE  TIME  OF  THE  ALGORITHMS. 

The  analysis  done  here  holds  for  both  Algorithms  1  and  2  (except 
that  they  differ  in  the  parameter  <ymin  defined  below).  We  assume  here 
the  terminology  of  section  4. 

Intuitively,  in  both  algorithms,  the  ASK  or  RESPOND  phrases  take 

0(v)  time  each.  In  the  worst  case  of  the  non-insisting  algorithm,  it 

requires  0(v)  expected  executions  of  the  ASK  phrase  to  choose  any  given 

willing  neighbour,  if  the  set  of  willing  neighbours  is  0(v).  Given  that 

a  given  neighbour  is  chosen  and  he  is  willing,  communication  will  be 

achieved  with  probability  bounded  below  by  a  constant.  Hence,  we  expect 

2 

the  average  time  of  response  of  the  non-insisting  algorithm  to  be  0(v  ). 

On  the  other  hand,  in  the  asking  phase  of  the  insisting  algorithm 
we  ask  a  specific  neighbour  and  we  have  a  constant  probability  to 
communicate  with  him,  if  he  is  willing.  Thus,  the  expected  total 
number  of  phases  will  be  0(1)  and  so  the  expected  response  time  of  the 
insisting  algorithm  will  be  0(v)  in  the  worst  case. 


KHfHF  'r=*  f  *l!V  It  :i -  ■ 


A  formal  analysis  follows. 

Let  I  be  a  time  interval  of  length  *2cnar  (i.e.,  at  least  two 

ItiflX 

phases),  such  that  i*^j  and  i,j  both  tame  on  I.  Fix  some 

time  t  >0.  Let  r  be  the  global  system  history  up  to  t, 

derived  from  oracle  of and  "luck  up  to  time  t"  as  defined  in  Section  2B, 

Note  that  (u/,Tt)  essentially  specifies  everything  of  the  system's 
immediate  future  except  the  pollers'  "luck"  L^,  for  time  t'  >  t.  For  all 
i,j€II  let  a±  .  be  the  probability  that  the  poller  of  i  is  answered 

by  j  some  time  within  b,  given  i  is  in  a  watching  window  for  j  during  time 

interval  A  starting  at  time  t,  and  assuming  both  i,j  are  tame  on  A  ,  where  A 
In  the  following  analysis,  we  assume  constants  0nan ,  d”1**  such  that 
(*)  0  <  omin  <  a.  <  omax<  1  for  all  i,j,  6n,  for  all  t  a  0, 

all  oracles  and  any  global  system  history  Tt,  consistent  with 

ui . 

In  the  next  section  we  show 

THEOREM  5.1:  For  Algorithm  1,  we  may  let 
min  _  1 


THEOREM  5.2:  For  Algorithm  2, 


x  1  _ndn  c, 


where 


c.  «  i  -  -L  (l-e  R)  . 
CR 


Also,  for  simplicity  we  let  amax  *  1  for  both  Algorithms  1,  2.  Let 

X  . (of,T  )  be  the  probability  that  the  poller  of  i  is  answered  by  j 
i  j  t 

some  time  within  A  c  I  and  i  is  in  a  watching  window  for  j  during 
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time  interval  A  starting  at  t,  where  A  c  I . 

From  the  definitions  of  ^  *  oi  ,rfc)  and  X^ (  ^ ,T t>  it  immediately 
follows  that 


Xij  (  ^*rt)  *  2  *  ai j  ^  for  ^Xgorithm  2 

^i;j(c^*rt)  -  j  •  j-  *  for  Algorithm  1 

where  d^,  *  D^  (the  outdegree  of  i)  at  the  time  instant  t’  <  t  at  which 
i  selects  a  neighbour  to  ask. 

Since  by  Al,  1  6  dfc,  <  v,  it  follows  that 

omin  <  X„  (  ^,Tt)  <  ^  amaX  for  Algorithm  1 


and 


J  amin  <  X^  (  ^,1^)  <  i  amaX  for  Algorithm  2  . 


Let  W  be  the  class  of  oracles  j/ for  which  the  outdegree  d^,  is  set 
equal  to  v  for  all  nodes  i  in  G^,  and  for  all  instances  t'. 


Proposition  5.1.  The  response  time  of  Algorithm  1  increases  with 
increased  requests  to  communication. 

Proof.  The  probability  that  a  specific  process  is  chosen  in  the  ASK 
or  RESPOND  phases  decreases  monotonically  with  the  number  of  processes 
to  which  the  process  executing  ASK  or  RESPOND  is  willing  to  communicate,  o 
By  Proposition  5.1,  the  class  of  oracles  Ogives  an  upper  bound  in 
the  response  time  of  the  system,  since  adding  requests  to  communicate 
cannot  decrease  the  response  time. 

For  each  oracle  o 4$.  with  Algorithm  1, 


% 


* 

f 


-  * c-‘ ' 
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Let  us  define 


and  let 


Also  let 


.min  1  min 
x  -27  0 


.min  1  min 
A  ■  —  o 


(for  Algorithm  1) 


(for  Algorithm  2)  . 


max  m  _1_  gmax  (for  Algorithm  1  and  oracles  9#  ) 


2v 
1  max 

"  2  a 


(for  Algorithm  1  with  all  oracles  but  those 
of  and  Algorithm  2  for  all  oracles) 


For  all  i,j€II  let  P  (k/ (  ai  ,Tt) )  be  the  probability  it  takes  exactly  k 
phases  for  process  i  to  be  answered  by  j,  given  that  i  j  for  a  time 
interval  A  starting  at  t,  such  that  Acl. 

Lemma  5.1:  For  any  oracle 

<  ptJ<k/^.rt))  <  x™,t(i-x”in)k"1 

Proof,  it  suffices  to  observe  that  the  process  of  i  be  answered  by  j  is 

a  geometric  stochastic  process,  with  success  probability  bounded  by 
...min  .max. 

I A  f  A  J  •  D 

By  using  Lemma  5.1  and  calculating  the  mean,  we  get 
Lemma  5.2: 


,min 


.max 


(X”“>2 


<  mean  (k)  < 


(Xmin)2 


By  Lemma  5.2  and  known  expressions  for  the  tail  of  the  geometric. 
Lemma  5.3:  Ve  0  <  e  <  1,  Prob{k  >  k„„,(E>}  <  E, 

- - - -  iiulX 


where 


log  (1 


min 


£)  -  log  X 


max 


log  (1-X,flin) 


Let  cv  be  the  number  of  steps  required  for  each  phase.  Lemmas 


5.1  and  5.2  imply 
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Theorem  5.3s  If  t  is  the  response  of  the  system,  mean(t)  <  cv  r 
-  max 

XmaX/(Xmin)2  and  if  i(e)  is  the  e-error  response,  t(e)  <  cv  r  •  k  (c) . 

max  max 

Finally,  Proposition  (5.1)  and  the  above  theorem,  together  with  theorems 
5.1  and  5.2,  imply  the  following  corollaries,  as  claimed  in  Section  2E. 

By  using  the  value  of  AmaX  for  Algorithm  1  and  oracles  #and  the 
value  of  An'in  for  Algorithm  1,  we  get 


max 


mean(t)  <  2cv  r 


max  ,  min.  2 

(o  ) 

By  using  omaX  ■  1,  =  1/4  we  have 

2  2 

Corollary  5.1:  meanly)  <  32cr  v  »  0(v  )  for  Algorithm  1.  Since 
-  max 


,  /  e  min\  .  1  i 

log  (  2v  a  I'  l09  2V° 


1  _max 


k  (e) 
max 


log(l  -  I  0^) 


-*■  8v  log  — 
^  e 


for  v  »  o. 


Corollary  5.2:  t  (e)  <  8c  rmaxv2log  *  0  ^v2log  J 

for  Algorithm  1. 

Also,  by  using  the  derived  An'*n,  A1”8*  for  Algorithm  2, 


Corollary  5.3:  For  Algorithm  2,  , 

3 


mean  (y)  <  16  maX  — — 

rmin  (c*) 


2 


v 


0  (v) 


Also,  we  get  for  Algorithm  2, 


log  ( |  omin  -  log 
‘  (log  1  -  i  o 
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Ml 

log  (l  - 


r 

max 


1  rmln 

8  r  c 
max 


) 

■) 


Corollary  5.4:  t(e)  <  cr _ k _ (e)v 

- - —  max  max 

■  0  (v  log  i  ) 


6.  ESTIMATION  OF  omin 

6.1  Analysis  of  the  Non-insisting  Algorithm  1 

Algorithm  1  is  non-insisting :  if  i  €  II  is  in  its  asking  mode, 
it  chooses  to  ask  a  random  j  from  the  set  of  pollers  to  which  i  is 
willing  to  comnunicate.  Observe  that  the  probability  of  i,j  communicating 
monot.onically  increases  with  the  rate  of  the  responding  process  and 
decreases  with  the  rate  of  the  asking  process.  However,  if  both  i,j  are 
willing  to  communicate,  both  of  the  i,j  will  attenpt  to  establish 
communications.  Because  of  this  symmetry  in  the  way  processes  ask,  it 
follows  that  the  worst  case  j {  is  when  the  oracle  sets  the 

rates  i,j  equal,  but  not  necessarily  constant  (recall  that  by  our  model 
of  Section  2B,  .V determines  the  rates  {Rfc|t  >  0}  at  time  0  and  u/ cannot 
influence  the  probabilistic  choice  of  processes).  Let  cRv  +  2  be  the 
fixed  number  of  steps  between  phases,  as  given  in  Lemma  4.2.  Let  x  be 
the  number  of  steps  by  which  i  executes  each  phase  before  j,  where 
0  <  x  £  cRv.  Let  S (S' )  be  the  event:  j  answers  i  is  in  its  asking 
mode  after  (before)  j  assuming  i,j  both  willing  to  communicate. 


Then 
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°ij( 


~  Prob(s)  +(1  -  j  Prob  (S) ) 
~  Prob  (S’) 


since  Prob(j  is  in  answer  mode)  *  1/2  and  j  may  answer  i  either  in  j's 
current  phase  (in  the  phase  where  j  got  the  question)  or,  at  most  in  the 
next  phase  of  j .  Note  that  the  length  of  the  watching  window  of  i 

does  not  allow  for  more  than  two  phases  of  j  ( see  Figure  1) . 

Let 

f (x,y)  «  1 


-H 


x/c. 


We  have  Prob(s)  «  f(x,dt,)  where  d^,  is  the  outdegree  of  j  at  time  t'  when  j 
selected  the  v  neighbours  to  answer.  Let 

g(xfy)  »|f(x,y)  +  (1  *  jf(x,y))  •  y  f  (cRv- x- 7,y) . 

Then,  since  g  is  monotonically  decreasing  with  y  and  x  i3  bimodal, 

CT..(a/,r  )  >  MIN  {g(x,y) } 

3  r  0<x<cRv 

0  <y  <  v 


6.2  Analysis  of  the  Insisting  Algorithm  2 

Algorithm  2  is  insisting :  poler  i  always  asks  that  j  which  if  the 

target  of  the  first  edge  of  departing  from  i.  The  worst  case  is 

where  the  oracle  .Vsets  the  speed  of  the  asking  process  to  be 

1/r  .  ,  and  the  speed  of  the  answering  process  to  be  l/r  .  Let  S  and  x 
min  max 

and  f  (x,y)  be  as  in  6.1.  Because  of  the  random  wait  in  the  beginning  of 
each  phase,  x  now  can  be  any  step  in  [0,cRv]  with  equal  probability,  and  so 


.4. 


figure  1 


Case  j 

probabilistic 

choice  of  mode  *  in  *nsWer  mode 


41 -  — - _ *  I 

watching  window 
of  i 

Case  2 

j  is  in  answer  mode 


NOTE: 


'  -  -  a  phase 


- 
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Prob{j  is  in  answering  mode} 

■  Prob{ j  is  not  waiting}  •  Prob{j  is  answering/j  not  waiting} 

>1.1.1 

2  2  4  * 

We  have  (  by  considering  only  case  1  of  Figure  1) 

V-'-'V  *  j  Prob  (s)  •  Prob(i's  window  was  large  enough 
to  get  the  answer).  However,  due  to  the  uniformly  distributed  random 


choice  of  x 


Prob (S)  >  l  f(y,d  ,)  •  Prob(x»Y) 
Y«*0  * 


*  l  •  -i- 

Y-0  CRV 


V41  J 

CRV  "  C 


i(-Hr') 


>  c'  for  v  »  0 


where 


C  -  !  -  JL  1  -  e  R 

CR 


Also,  Prob(i's  window  is  large  enov.gh  to  get  the  answer) 

^  ^Srmin  dt 

' o  c_v  r 

R  max 

rmin 

*  — -  for  v  »  0, 

max 


hence 


1  rmin 


aij  ^  ^  *^t^  *2  r 
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7.  CONCLUSION 

We  have  provided  two  real  time  implementations  for  the  DCS  system. 

A  key  assumption  on  our  time  analysis  is  that  processes  have  to  be  tame 
during  attempts  to  communicate,  but  at  other  times  processes  need  not 
be  tame.  This  improves  a  previous  version  of  this  paper  fReif,  Spirakis, 
1981a],  where  we  required  processes  to  be  tame  at  all  times. 

A  referee  has  suggested  a  modification  of  our  algorithms  which  may 
be  of  practical  use  in  speeding  up  the  expected  time  response  in  some 
practical  cases.  The  modification  presumes  that  the  connections  graph 
has  fixed  valence  (otherwise,  an  infinite  number  of  variables  per  process 
is  required) .  The  idea  is  to  allow  each  process  to  have  additional  flag 
variables  which  indicate  to  other  processes  its  willingness  to  communicate 
with  them.  (We  had  presumed  that  the  set  Ei  can  only  be  read  by  process  i), 
so  the  idea  requires  additional  flag  variables.  The  modified  algorithms 
will  have  worst  case  performance  identical  to  those  given  in  our  paper. 

In  a  further  paper,  [Reif,  Spirakis,  1981BJ ,  we  have  relaxed  our 
assumption  of  tameness.  In  that  paper  we  require  only  bounds  on  the 
relative  acceleration  of  ratios  of  speeds  of  neighbour  processes.  We 
propose  there  synchronisation  algorithms  which  have  relative  real  time 
response,  where  communication  is  established  with  high  probability 
between  any  pair  of  processes  within  constant  number  of  steps  of  the 
sloueat  process.  Hcwever,  these  algorithms  are  less  efficient  than  those 
given  in  this  paper.  Also,  we  are  applying  our  synchronization  techniques 
to  ADA  for  a  relative  real  time  implementation. 
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APPENDIX 

A  REAL-TIME  IMPLEMENTATION  OF  CSP 


[Hoare,  1978]  introduced  a  concurrent  programming  language  CSP 
(for  Communicating  Sequential  Processes) .  The  CSP  language  is  notable 
for  the  elegance  of  its  synchronization  constructs!  They  are  powerful 
and  yet  simple.  [Bernstein,  1980]  describes  an  extension  of  CSP  which 
allows  both  input  command  and  output  commands  as  guards.  Here  we  briefly 
describe  CSP  with  Bernstein's  extension  and  present  a  real-time  implemen¬ 
tation  of  the  synchronization  constructs. 


CSP  Synchronization  Constructs 


The  relevant  aspects  of  CSP  concern  its  process  structure  and  com¬ 
munication  mechanisms.  Concurrent  execution  of  processes  P^,  Pj,  . ..,  Pn 


is  denoted 


[Pill  P2I|  ...||  PnJ  . 


Each  process  has  its  own  set  of  variables  which  are  inaccessible  to  all 
other  processes.  The  comnunication  primitives  are  the  output  command 
Pjlu  that  requests  that  P^  receive  the  value  of  u  and  input  command 
P^?x  which  requests  that  P^  send  a  value  which  is  then  assigned  to  x. 

There  are  two  relevant  compound  statements.  The  alternative  statement 


tGi  *  et  o  o2  *  c2  d  ... 
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contains  guards  G,  ,...,G.  and  command  lists  C,  ,...,C..  Each  guard  consists 
of  a  list  of  elements  which  may  be  a  sequence  of  booleans,  followed  by  at  most 
one  input  command  or  (in  Bernstein's  extension  of  CSP)  an  output  command. 

The  execution  nondeterminately  chooses  a  guard  G^  which  is  satisfied 
(to  test  that,  it  executes  each  element  of  G^  from  left  to  right)  and  then 
executes  the  corresponding  command  list  C^..  If  no  guard  is  satisfied,  the 
alternative  statement  fails.  The  repetitive  statement 

*IGi*c1o...oGk..ckj 

results  in  the  repeated  execution  of  the  alternative  statement 
IG^  -*■  o  . . .  o  g^  •+■  C^]  ,  until  no  guards  are  satisfied. 

Note  that  the  crucial  problem  in  implementing  CSP  is  to  synchronize 
executions  of  input  commands  P..?x  by  process  P^^  with  output  commands  P^u 
by  process  P..  so  that  the  value  u  is  transmitted  to  x. 

It  is  very  easy  to  implement  CSP  by  DCS.  (In  fact,  this  was  the 
original  motivation  for  our  work  on  DCS).  Let  e  be  a  system-wide  constant, 
which  may  be  fixed  to  any  arbitrarily  small  constant  on  the  interval  (0,1) . 

We  assume  a  real  time  DCS  implementation  with  e-error  response  time  x(e). 

Let  v  be  the  maximum  number  of  guards  appearing  in  any  alternative  or 
repetitive  statement;  we  assume  that  v  is  constant  relative  to  the  total 
number  n  of  processes.  We  also  assume  that  the  length  of  the  guard  lists 
is  bounded  by  a  small  fixed  constant.  We  also  assume  all  processes  reliably 
execute  their  programs  and  satisfy  assumptions  Al  and  A2. 

Our  CSP  implementation  is  real-time  in  the  sense  that  there  exists  a 
positive  integer  Z  (which  is  independent  of  the  number  of  processes  r.) 
such  that  if  in  some  alternative  or  repetitive  statement  S  some  guard  G 


is  continuously  satisfied  for  a  time  interval  A  of  length  >£  and  if  the 

processes  of  G  and  the  process  executing  the  statement  are  tame  on  A, 

than  the  command  list  associated  with  some  satisfied  guard  is  immediately 

executed  with  probability  >  1-E  and  otherwise,  a  failure  exit  Lb  always 

made  immediately  after  a  time  interval  of  length  £.  Therefore,  we  allow 

a  failure  exit  with  probability  <£,  even  though  some  guard  may  be  satisfied. 

To  attempt  to  execute  an  output  command  PJu  in  process  P^ ,  P^ 

sets  Pr- *Py  indicating  Pi  is  willing  to  communicate  with  P^.  Also,  to 

attempt  to  execute  an  input  comand  P^?x  in  process  P ^ ,  P,.  sets  P^->P^. 

If  successful  communication  is  established  by  P.  and  P.,  the  process  P. 

*  3  1 

immediately  transmits  value  u  to  variable  x  in  P^;  and  immediately 

thereafter  P.  sets  P.-/*P.  and  P.  sets 

i  i  D  D  D  •  i 

An  alternative  or  repetitive  statement  S  may  contain  the  execution 

of  one  of  several  guarded  input  commands  and  output  commands,  say 

where  s<v.  To  execute  the  statement  S,  P.  first  executes  the 
Is  i 

booleans  appearing  in  each  guard.  Let  R  be  the  set  of  processes  appearing 

in  those  guards  of  S  all  of  whose  booleans  evaluate  to  true.  P^  must  set 

P.— *P .  for  each  P.Gr  for  a  time  interval  of  length  £*=T(e).  At  the 
a-D  :3 

first  time  that  an  appropriate  communication  is  established  between  and 

some  willing  process  P^  €  R,  P^  must  immediately  set  P^-/d  P ^  ,  for  all 

P . , €  R  and  then  P.  must  execute  the  command  list  associated  with  the  now 
j  i 

satisfied  guard  in  the  statement  S.  Otherwise,  if  no  appropriate  communica¬ 


tion  is  established  within  time  T(e),  p^  must  then  exit  the  statement  S 
with  failure.  Note  that  the  probability  of  an  incorrect  failure  exit  is  <£. 


